CGIAR Evaluation Framework and Policy

They both reinforce the use of evaluations for learning, to support decision-making and accountability in One CGIAR, as recommended in the 2019 CGIAR MOPAN Assessment. With CGIAR management and the community of the CGIAR monitoring, evaluation and learning community of practitioners (MELCOP), the Framework and Policy:

• Enhance attention to quality assurance dimensions for the performance and process evaluation in CGIAR, irrespective of the commissioner;
• Enhance the capacity of relevant staff who manage evaluations, and of those who have a role in discharging the evaluation practice within the CGIAR system.
• Reinforce and strengthen the complementarity between evaluation and other PRMF-related functions in One CGIAR.

The evaluation framework and policy guide the conduct of all performance and process evaluations in One CGIAR, including Platform evaluations and evaluations of non-pooled projects.

Impact assessments are explicitly out of the scope of both the CGIAR Evaluation Framework and Policy. All related definitions are provided in the MELIA glossary. Consistent with the Evaluation Policy’s pillar two (of five) “A Holistic and Consistent CGIAR-wide Approach to MEL” Evaluations collect and/or use the best available primary and secondary data supplemented by other data as necessary” which would include evidence from impact assessments.

A fit-for-purpose evaluation policy is needed to support the evolving needs and demands of One CGIAR. As a part of the iterative policy revision approach vetted with SIMEC in 2021, the evaluation function under the Independent Advisory and Evaluation Service conducted a survey of CGIAR MEL Community of Practice (MELCOP) in March/April 2021 to gather their feedback as key co-users of the policy and framework. Their responses to the multiple-choice questions indicated the desired top objectives of the evaluation policy:

• To provide guidance on evaluation principles, approaches and criteria; and requirements of conduct and use of evaluations by programs/Centers;
• To provide guidance on requirements for conduct and use of evaluations to CGIAR Centers and Initiatives; and
• To enhance use of evaluation for decision-making.

Stage-gating decisions can be supported by ‘evaluability assessments', which help identify potential bottlenecks and solutions to allow for adjustments towards evaluation readiness. The choice of evaluation criteria (during an evaluability assessment or scoping for an evaluation) would align with the stage-gate at which decisions are being made. While the ‘Effectiveness’ criterion would be key for stage-gating, such criteria as ‘Relevance’ or ‘Sustainability’ may not be. Overall, stage gating is out of scope for the Evaluation Policy, and the related guidelines are being elaborated in relation to the PRMF and the Evaluation policy.  

The Evaluation Framework, Policy, and supporting guidelines are all aligned with the PRMF and stage gating process as detailed in the CGIAR Technical Reporting Arrangement (Pages 16-17). Evidence from evaluations (including independent evaluations per the approved independent evaluation workplan) informs adaptive management and business cycle assurance stage gating processes. The evaluation guidelines which provide implementation support, are not meant to be prescriptive; they are to be tailored to meet the specific circumstances, the unique needs, and decision gates of an Initiative (or intervention). 

The 2030 Research and Innovation Strategy is guided by the new performance and results management framework (PRMF) 2022-2030 to feed in the data and evidence from key areas of work. While the implementation of the 2022-2030 PRMF and its associated framework support an enabling environment and provides the conceptual framework for effective measurement, learning, and accountability from performance and results, the implementation of the PRMF thereof is not within the remit of the Evaluation Framework and Policy.

The Evaluation Framework and Policy are rooted in the PRMF, as it guides accountability to funders. Reliance on a theory of change, a theory-based approach, is at the core of research and evaluation. Refer to section 3.2, page 6 of the Evaluation Framework: “Theories of change describe the pathways to impact—which can be complex, intersecting, and often nonlinear—drawing on insights from the social sciences, including economics and international relations theory.

ISDC reviews are not evaluations as defined under the Policy and Framework. The ISDC developed the quality of research for development (QoR4D) framework to review proposals, and its elements align with the evaluation criteria.

The Evaluation Framework assumes that responsibilities pertaining to evaluation will be carried out by the CGIAR governance, management, the evaluation function of the Independent Advisory and Evaluation Service, and units within the operational structure that pertain to monitoring, evaluation, and learning (MEL) as detailed in current governance frameworks, charters, and similar documents (see Evaluation Framework page 1).

The reporting lines of MEL professionals would need to be clarified in terms of reference of the CGIAR’s Project Coordination Unit (PCU) and Portfolio Performance Unit (PPU) within the Research Delivery and Impact Division.

The aim of the CGIAR-wide evaluation practice is to produce evaluations that provide accountability, support decision-making and learning (See also response to a related question on PRMF above). The standards and principles provide a point of reference for the professionalism of research-for-development evaluation and dictate how evaluation is conducted in CGIAR. For technical aspects, yes, evaluation professionals do bear functional responsibilities to comply with a related frameworks and policies, including that of evaluation and Performance and Results Monitoring Framework (PRMF). Of the five pillars of the evaluation Policy, two most related ones are:  

1.   Evaluation Embedded in CGIAR Assurance and Management.

2.   A Holistic and Consistent CGIAR-wide Approach to MEL.

CGIAR works at the intersection of science and development. The Independent Advisory and Evaluation Service (IAES) comprise the Independent Science for Development Council (ISDC), the Standing Panel on Impact Assessment (SPIA), and an Independent Evaluation Function implemented at the request of CGIAR System Council. Together IAES provides external, impartial, and expert advice related to strategy and positioning, program evaluation, and impact assessment. 

The evaluation criteria of this Policy adhere to the 2019 OECD Development Assistance Committee (OECD/DAC) evaluation criteria and reflect the characteristics of research for development in CGIAR’s context, consistent with the quality of research for development (QoR4D) framework elements, elaborated by ISDC. They facilitate CGIAR System-wide agreement on the nature and assessment of the quality of science, subject to evaluation (see forthcoming guidelines to evaluate QoS).

One of the three functions of the Project Coordination Unit (PCU) includes; providing standards and capacity building on project and portfolio management, including MELIA, best practices, and procedures.

Everyone in CGIAR, including MEL professionals, Initiative Leads, research scientists, Leadership et al is responsible for the delivery and implementation of the policy and framework.

The policy articulates the roles and responsibilities for the key stakeholders; CGIAR Group Operational Units including MEL professionals, CGIAR, Leadership et al. The Evaluation Framework and Policy underscore the importance of leadership commitment to a robust evaluation practice that delivers value to CGIAR and its stakeholders. For leadership, this implies appropriating adequate resources to Monitoring, Evaluation and Learning (MEL) structures and systems and engendering a results-based culture that promotes the effective use of evaluation for accountability, to promote learning and steer decision-making.

One of the 15 standards and principles of the evaluation framework is “Mutual accountability: In CGIAR, expectations for evaluation are matched with adequate investments in requisite financial and human resources. The capacity and systems for data collection and real-time information underpin mutual accountability” (Evaluation Framework pg. 4).

Decisions to conduct an evaluation are driven by CGIAR funders, and management, depending on the type and timing of evaluation, and needs within the business cycle (mid-end of intervention, platform, etc.)- see CGIAR reporting arrangement. An evaluability assessment would help decide whether to conduct an evaluation and with what scope.

To support the implementation of the revised evaluation policy and new evaluation framework, the Evaluation Function within the Independent Advisory and Evaluation Service has co-developed guidance; evaluability assessment, evaluating Quality of Science, et al. Through engagement with MEL COP, needs will be identified and addressed by relevant bodies. 

Evaluation criteria serve as the basis upon which evaluative judgments are made to assess CGIAR interventions and processes to provide accountability, support decision-making, and learning. The Policy’s definition of impact and effectiveness encompasses both the OECD DAC criteria and Quality of Research for Development (QoR4D) elements, as applicable to the CGIAR context. Thus, the ”Impact” and “effectiveness” evaluation criteria are within the scope of evaluations covered by this Evaluation Policy and Framework.

The PRMF applies beyond 2024, as does the Evaluation Framework and the Policy. The process of management engagement and response to evaluations will facilitate sustainability and ensure that appropriate, agreed-upon and joined-up institutional vehicles are in place through which follow-up actions will be implemented.